Open
Conversation
Announce Doc Detective support in Promptless as a product update.
InlinePizza
requested review from
adit-chandra,
frances720,
mattlink and
prithvi2206
as code owners
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
hawkeyexl
suggested changes
Mar 13, 2026
hawkeyexl
left a comment
hawkeyexl
left a comment
There was a problem hiding this comment.
I like it overall, but it doesn't address why you might need Doc Detective is you already have Promptless, how they're two sides of the same coin. Promptless reduces drift, but Doc Detective verified that any content, human-created or AI-generated, is accurate.
|
|
||
| Back in August 2024, Manny Silva was one of the first technical writers from whom we ever got feedback about Promptless. Back then, he'd already been working on Doc Detective for years, and it seemed like Promptless and Doc Detective would work perfectly together. | ||
|
|
||
| **Today, we released Doc Detective support in Promptless**, meaning that Promptless can create/update Doc Detective tests automatically and fix documentation or escalate software issues whenever Doc Detective tests break! |
There was a problem hiding this comment.
Use "fail" instead of "break". When features break, tests fail.
|
|
||
| ## Use case 5: Authenticated flows | ||
|
|
||
| Most useful documentation covers features behind a login. Doc Detective handles this with cookie management (`saveCookie` / `loadCookie`) and has a Clerk integration for injecting test tokens. You can test documented procedures for authenticated features without manual login steps. |
There was a problem hiding this comment.
We don't have an official Clerk integration, but I'm working on that feature request we discussed.
|
|
||
| It handles the messiness of real web applications too. You can configure wait conditions that monitor network activity and DOM mutations, so the test only proceeds once the page has actually finished loading. Useful when your single-page app is still fetching data three seconds after navigation. | ||
|
|
||
| The screenshot capability goes further than just capturing images. Doc Detective compares screenshots against reference images pixel by pixel. You set a variation threshold (say 5%), and if the current state of the UI has drifted beyond that, the test flags it. So when someone changes the product and the docs still show the old screenshots, you find out from your test suite, not from a confused user filing a support ticket. |
There was a problem hiding this comment.
Repeated in its own section later on.
|
|
||
| A specification contains tests, each test is a sequence of steps, and each step is an action: | ||
|
|
||
| ```json |
There was a problem hiding this comment.
This whole object is completely wrong. We haven't used the "action" key since v2.
| } | ||
| ``` | ||
|
|
||
| You can also embed tests directly in Markdown as comments, or use automatic markup detection. Doc Detective can parse instructions like "Go to [the dashboard](https://app.example.com/)" and generate the corresponding test action from the Markdown itself. |
There was a problem hiding this comment.
You're likely going to need code syntax here so the markdown doesn't render.
|
|
||
| ## Where does it fit in your workflow? | ||
|
|
||
| Doc Detective runs from the command line. Put it in your CI/CD pipeline. Every time someone changes the docs or the product, the doc tests run. Failures show up alongside your code test failures. The coupling between docs and product becomes something you actively monitor, not something that silently degrades. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan